Amiga Tools 4/Amiga Tools 4.iso/tools/netzwerk/tcp-ip/usno/docs/interex.paper

Text File | 1996-02-26 | 16.5 KB | 335 lines

                USING BSD IPC TO SYNCHRONIZE SYSTEM TIMES

                               Wendy King
                            Richard Schmidt
                         U.S. Naval Observatory
                         Time Service Department
                       34th & Massachusetts Ave., NW
                        Washington, DC  20392-5100




The Time Service Department of the U.S. Naval Observatory maintains a
number of Hewlett Packard precision cesium-beam frequency standards in a
system providing the USNO Master Clock.  Comparisons are made in
nanosecond units and are monitored on a network of HP computers whose
system clocks have in the past differed by MINUTES of time!  Aside from
the obvious irony there are very good reasons for wanting uniform time
in an ensemble of computers.  Processes distributed across a network of
machines may need synchronized access to shared files.  File management
across NFS mounts, as with the makefile utility, may depend on
reasonably close system time agreement.  Time synchronization becomes
even more important when independent machines must acquire and share
real-time data.

But the crystal clocks in workstations are inherently unstable.  Drift
rates of up to +/- 5 seconds per day are reported for the HP9000/300s.

We have developed a simple clock synchronization scheme based on the
concept of a "time server" host which is kept on Master Clock time via a
serial link, and an unlimited number of networked client machines which
each set their system clocks to server time over a TCP/IP network.  The
time setting process is auto-scheduled as a daily task with cron.

Hewlett-Packard 9000's and 1000's, Sun workstations, and DEC hosts,
among others, support a system of interprocess communication known as
the "Berkeley Software Distribution."  BSD IPC enables process
communication via addressable sockets.  Data packets can be written to
and read from sockets much like normal files.  Internet stream sockets
are specifiable by the Internet address of a host and the number of a
TCP or UDP port, enabling communications across local and wide area
networks.

Our implementation of time syncronization uses Berkeley stream sockets,
providing reliable, sequenced, flow-controlled communication of byte
streams which are always received in the order that they are sent (2).
TCP provides the underlying transmission protocol.

Networked time synchronization originates from a designated server host
whose system clock is set to an external source.  At the U.S. Naval
Observatory in Washington, an HP9000/835 and two HP1000/A990's obtain
time from redundant serial links to USNO Master Clock systems 1 and 2.



On all our systems, the time is set to within one second, which is well
within the accuracy needs of cron-scheduled processes. (Cron schedules
only to one minute precision).  On the HP1000 systems, the time is set
using the Master Clock link during the daily, automatic, re-boot, and
the accuracy we achieve is less than 1/4 of a second.  Our A990 systems
are used for real-time data acquisition and instrument control, and we
can schedule processes to a tenth of a second accuracy.

A server process called "timesrv" resides on the server hosts.  It
provides current system time on request via the TCP/IP network.  On the
Unix system timesrv relies on the internet daemon "inetd" to be alert to
activity on timesrv's particular socket port.  On the RTE systems, the
internet daemon "inetd" only watches for Mail/1000 connection requests,
and any NET IPC user applications that may have been installed.  The
"timesrv" program on RTE must do the work of "inetd" for itself.

Any networked client wanting the server's time of day simply requests a
connection to timesrv and reads back current time.  The client then
calls the system stime() routine to set its system clock to the received
time.  We did not develop a set-time program for the HP1000 systems
because they have their own serial link to the Master Clock.

An auxilliary program, "timecheck", enables any user to check his system
time against that of the time server.  Corrections are not made for
network propagation or response delays.  We have tested Internet delays
between Washington, D.C and Richmond, Florida, and we have never failed
to achieve 1-second accuracy.  The initial connection request can take
several (sometimes many) seconds, but once the connection is
established, the time packet (4-bytes) is consistently received within
one second.  We have also never failed to get one-second accuracy
between any of our systems on the USNO LAN.

Time transfer from the time server host to a client is initiated at the
client end.  The programs "time_set" or "TimeCheck" run on the client side,
communicating with the program "timesrv" on the server host.  The client
program performs nearly all of the work, which is as it should be.  It
must request a connection to the server, receive back a 4-byte time
value, shutdown the connection, and set the local system time or display
the difference between the server's time and the local system time.  This
is done with appropriate calls to the BSD IPC library.

The BSD IPC library is written in C.  It is a library of functions which
can be called to interface with the OSI ISO transport layer.  As long as
the client or server is written in C, using the BSD IPC library does not
present any special problems.  For HP1000 systems which do not have a C
compiler, FORTRAN or Pascal may be used, and HP has provided routines
and code examples to handle pointers and structure data types which are
not intrinsic, particularly to FORTRAN.  We used FORTRAN for the client
and server on our RTE systems.


The HP NS product includes examples of programs in C, FORTRAN, and Pascal,
as well as the necessary "include" files with the BSD data types defined.
To maintain consistency, our RTE systems all have a global /include
directory which contains all the files in the NS product that were
supplied in the /ns1000/include sub-directory.  All the data types
referred to in this paper can be found in the /include/sockets.ftni or
/include/sockets.h files.


USING BSD UTILITIES AND CALLS

Function "gethostbyname" is called with a hostname argument known either
via a local hosts file, a domain nameserver, or through the yellow pages
(Network Information Services).  Returned is a pointer to a "hostent"
structure, which contains among its elements the network (IP) address of
the specified host.

The FORTRAN client program on the RTE system must use some special
tricks to dereference the pointer and extract the 32-bit value from an
array of 16-bit elements.  The value returned by gethostbyname is a
pointer to a structure.  Since FORTRAN does not allow structures,
offsets must be used to get the values from the structure fields.  The
IP Address, which is the value we need, is referenced by the first element
in an array of pointers in the fourth field of the hostent structure. (See
/ns1000/include/sockets.ftni).

In order to dereference pointers returned by the BSD IPC functions, the
entire data segment is declared to be an array.  The pointer is then
used as an index into the array to obtain the value.  As shown in Figure
1, the alias directive is used to name an array as a common block
starting from absolute address 0. This is followed by a COMMON
declaration of the same name, and the elements of this common block are
declared as an array.  FORTRAN does no array bounds check, so this array
can be indexed with any value. (See the FORTRAN 77 manual).

Since, in the case of the hostent structure, each field is a 16-bit
integer, X = mem(HostPtr + 4) gives the value of H_ADDR_LIST in X.  The
first element of this array is the pointer to the IP Address of the Time
Server Host, so Y = mem(x) returns the pointer to the IP Address.  Unix
systems declare this as a character pointer, and since the IP Address is
guaranteed to be word aligned, dividing it by 2 gives the word address,
(Z = Y/2).  The 32-bit IP Address could be accessed from two memory
locations, mem(z) and mem(z+1), but the double load instruction will
transfer a 32-bit quantity in one step, hence (IpAddr = dld(mem(z))).
The statement which puts this all together is shown in Figure 1. (See
/ns1000/examples/bsdclient.ftn).

Figure 1.  Handling pointers and structures in FORTRAN

$alias dld = '.DLD',direct
$alias /mem/ = 0
       Common /mem/mem(0:1)
       Integer mem
       Integer HostPtr
       Integer*4 dld
       Integer*4 IPAddr
       Integer*2  IHostName(32),IService(4)
       Character  CHostName*64 ,CService*8
       Equivalence (IHostName,CHostName),(IService,CService)

       i = TrimLen(CHostName) + 1
       CHostName(i:i) = Char(0)
       HostPtr = GetHostByName(ByteAdrOf(IHostName(1),0))
       IPAddr  = dld(mem(mem(mem(HostPtr + 4))/2))

       i = TrimLen(CService) + 1
       CService(i:i) = Char(0)

       ServPntr = GetServByName(ByteAdrof(IService,0),NULL)
       Serv_Port = mem(ServPntr + 2)

Next, "getservbyname" is called with argument "timesrv", the server's
time serving program.  Getservbyname merely searches the local
/etc/services file for the dedicated port associated with the service
"timesrv."  That port number is placed in a "servent" type structure,
and a pointer to that structure is returned.

In Figure 1, the last two lines show how the pointer to the service
port number is dereferenced in FORTRAN.  The value we want is in the
third field of the servent structure. (See /ns1000/include/socket.ftni).
Note also in Figure 1, the statements which place a null character at
the end of each string which is passed as an argument.  In C, all
character strings must be terminated with a null character.

A call to "socket" creates the socket as an AF_INET family member (for
Internet connections) and type SOCK_STREAM, a full-duplex byte stream
connection.

Function "connect" is called to make the actual connection.  Now the
remote time server springs into action.  An entry in the server's
/etc/services file links activity on timesrv's designated port with the
timesrv service:

     timesrv xxxxx/tcp myalias

where "xxxxx" is the assigned port number agreed between server and all
clients.  On the HP-UX systems, a related entry in /etc/inetd.conf enables
assigning responsibility for monitoring that activity with the inetd
program, eliminating the need for explicit daemons cluttering up the
process table.  On the RTE systems we are forced to clutter up the process
table, since the RTE inetd does not "do" berkeley sockets.  The format of
the /etc/inetd.conf entry on the HP-UX systems is

     timesrv stream tcp nowait root /usr/local/bin/timesrv timesrv

describing the service, its type of socket, protocol, mode of activity,
effective user id, path to the executable program, and name.

On the RTE system, a global directory /etc was created and the
"services", "hosts", "protocols", and "networks" files were ftp'd from
the HP-UX system.  This maintained consistency (everyone has all the
same entries) and if HP does provide further enhancements of ARPA
services for RTE, we will be ready!  No changes were required for these
files on the RTE system.  The inetd.conf file already existed in the
/system directory for Mail/1000, but is not involved with this
application since inetd on RTE does not service BSD at Revision 5.27.

On the HP-UX system, the time server program consists of only three
operations: a call to the Unix "time" function to obtain the system time
in seconds from epoch in the format of a long int , a "send" to pass out
the time, and a shutdown to close the connection.  As inetd communicates
via standard input (stdin), the file descriptor for send is zero.

On the RTE system, the time server program must do everything that the
HP-UX inetd does.  This includes a call to get a socket, a call to bind
the socket to the timesrv port number, a call to establish a listen
queue, and a call to accept connections.  The RTE time server is a
daemon.  It looks for connection requests, creates a new socket for a
requesting client, gets the system time, sends it through the new socket
to the client, shuts down the new socket, and returns to accept the next
request on the original socket.  We can break this infinite loop by
setting the breakflag and then initiating a request.

The RTE time server also moves itself to system session, and logs all
messages to a log file in the /system directory.  This proved helpful to
the frazzled system manager who needed to troubleshoot failures some
time after the event(s).  Every time the RTE time server services a
request, it checks the breakflag, closing its log file and terminating
gracefully if the breakflag is set.

As soon as the client sees a successful connection, it calls "recv" to
read the 4-byte time value (long seconds from the beginning epoch).  The
"ntohl" function is used to insulate the program from low-endian vs.
high-endian byte preferences.

At last the "stime" function is used to set the system time to the
received value.

In addition to the timesrv server program and the time_set client
program, a variant of time_set called "timecheck" follows all of the
steps of time_set, but rather than setting local system time, it simply
reports the difference between the local time and the server's time.

The HP1000 client TimeCheck program approaches the connection process a
little differently.  Since this program is typically run interactively,
blocking mode (the default) caused problems when requests did not
complete correctly.  So, the TimeCheck client uses non-blocking mode on
its socket which necessitates the use of the Select() function to check
for successfull completion of requests such as Connect() and Recv().
The Select() function is the only one which accepts a user defined time
out value, allowing the user's program to handle any event and continue
(or terminate) gracefully.

In our implementation of these programs, our HP-UX clients schedule
their timesetting requests via "cron".  It should be noted that prior to
changing the system time, cron must kill itself on the client and also
on any cnodes clustered to it.  A shell script which performs this
suicidal act and then returns cron to life is included in the source
distribution.

If you reset the system time on an RTE system, it is necessary to set up
a command file to "of" all user processes and to shutdown ns/1000 before
setting the time.  We tested the consequences of setting the RTE system
"on the fly" and found that time scheduled processes were randomly
adjusted or not, with no discernable pattern.

For RTE programmers (especially FORTRAN), there are a few documentation
"bugs" to watch out for in the BSD IPC Reference Manual (NS-ARPA/1000,
Volume 9, PN#91790-90060 E0891).

First, a library called HPC.LIB is required to link BSD programs.  We
found the only way to link a BSD program (examples included) without an
undefined symbols prompt was to create a .lod file for link which tells
it to search the HPC.LIB after the first EN.  To relocate errnodec.rel
along with the main program as suggested in the manual did not work.
Because of the order in which LINK searches libraries, and the order and
source of the calls in the BSD routines, the following .lod file was the
only one which avoided a LINK prompt for undefined symbols:

          * LINK command file to link BSD TimeServ program
          re timeserv.rel
          en
          se
          hpc.lib
          en

Second, calls which are looking for an event can execute in either
blocking (execute with wait) or nonblocking (execute without wait) mode.
Blocking means the call will not complete until the event it is looking
for occurs, or there is an error.  Nonblocking means the call completes
immediately, regardless of whether the event occurred, returning a status.
The manual incorrectly states that only nonblocking mode is currently
supported.  Both modes are supported, with blocking mode as the default.
The "Advanced Topics" section of the BSD Manual correctly documents
these modes.

Third (not really a "bug"), the use of the term "NULL" in the
documentation needs clarification for the non-C programmer.  Strings in
C must be terminated with a null character (i.e., Char(0)).  However, to
supply a null parameter in a function call, you must supply a zero
(integer) value.

Fourth (also not a "bug" but worth mentioning for non-C programmers),
the C language always indexes beginning with 0.  This means that
elements of structures begin with 0, and when the documentation says
that something is the "fourth" element of a structure, it is really the
fifth, but must be accessed using the pointer + 4 because the pointer is
pointing to the 0'th element of the structure.



NOTES:

(1) Hewlett-Packard Response Center, Atlanta.  Typical drift rate
observed on the HP9000/433s is +0.3 sec./day.
(2) Hewlett-Packard [users guide]